Refrigerator

ABSTRACT

A refrigerator provides visual information indicating a map of the refrigerator and technical operations of the corresponding regions. The refrigerator includes a voice recognition function using a proximity sensor. The refrigerator has input/output (I/O) management functions for foods stored therein. The refrigerator communicates with a peripheral device over a network.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application is a continuation application of prior application Ser.No. 16/406,727, filed on May 8, 2019, which is a continuationapplication of prior application Ser. No. 15/516,574, filed on Apr. 3,2017 and has issued as U.S. Pat. No. 10,436,498 on Oct. 8, 2019, whichis a U.S. National Stage application under 35 U.S.C. § 371 of anInternational application number PCT/KR2016/009390, filed on Aug. 24,2016, which was based on and claimed priority of a Korean patentapplication number 10-2015-0124702, filed on Sep. 3, 2015, in the KoreanIntellectual Property Office, and of a Korean patent application number10-2015-0184542, filed on Dec. 23, 2015, in the Korean IntellectualProperty Office, the disclosure of each of which is incorporated byreference herein in its entirety.

TECHNICAL FIELD

Embodiments of the present disclosure relate to a refrigerator.

BACKGROUND ART

Generally, a refrigerator stores foods or beverages (hereinafterreferred to as “stored goods”) in a fresh state for a long time usingcool air. The refrigerator generally classifies a storage chamber into arefrigerating chamber for storing the stored goods at a temperatureabove zero and a freezing chamber for storing the stored goods at atemperature below zero.

The refrigerator includes a compressor, a condenser, an expansion valve,and an evaporator in a machine room, such that the refrigeratorrepeatedly performs a cooling cycle for sequentially performingcompression→condensation→expansion→evaporation of a refrigerant.

The refrigerator can allow the inner space of the storage chamber to bekept at a freezing or refrigerating temperature using cool airheat-exchanged in the evaporation process of the cooling cycle.

Recently, due to improvement in the standard of living, storage capacityof the refrigerator is rapidly increasing to store more various and muchmore stored goods, such that functions have become more diversified forimprovement of user convenience.

DISCLOSURE Technical Problem

An object of the present disclosure is to provide a refrigerator forproviding visual information indicating a map of the refrigerator andtechnical operations of the corresponding regions.

Another object of the present disclosure is to provide a refrigeratorconfigured to include a voice recognition function using a proximitysensor.

Another object of the present disclosure is to provide a refrigeratorhaving input/output (I/O) management functions for foods stored therein.

Another object of the present disclosure is to provide a refrigeratorconfigured to communicate with a peripheral device over a network.

Technical Solution

The objects of the present disclosure can be achieved by providing arefrigerator including: a main body; a display provided in the mainbody; and a controller configured to control the display, wherein, if apredetermined condition is satisfied, the display is configured todisplay a refrigerator management screen image including a refrigeratormap.

The refrigerator may further include: a sensing part configured todetect an opening or closing state of a refrigerator door, wherein thecontroller, if the sensing part detects the opening or closing state ofthe refrigerator door, is configured to display the refrigeratormanagement screen image.

If the sensing part detects the opening or closing state of therefrigerator door, the controller may be configured to control thedisplay in a manner that constant temperature technology is visualizedand displayed on the display.

The display may be configured to display the refrigerator managementscreen image for visualizing/providing a state of the refrigerator.

The display may be configured to display the list of recommended foodsper indoor temperature in the vicinity of the refrigerator map.

The refrigerator may further include: a button part located in thevicinity of the display.

The refrigerator may further include a voice input part configured tocollect a voice command of a user.

The display may be configured to display a process for recognizing uservoice commands collected by the voice input part.

The refrigerator may further include a sensing part configured to detectuser approach in the vicinity of the refrigerator.

The controller may be configured to activate the voice input part on thebasis of user approach information collected by the sensing part.

In accordance with another aspect of the present disclosure, arefrigerator includes: a capturing part configured to capture bar codesprovided to foods or a packing container of the foods; a displayconfigured to display food information collected by the capturing part;and a controller configured to generate food information collected bythe capturing part, and control the display to display the generatedfood information.

The display may be configured to display at least one preset tag on apreset tag region provided to an upper end of a screen image of thedisplay, and may display images of foods stored in each chamber on afood display region provided at a lower end of the preset tag region.

The controller may be configured to control the display in a manner thata reminder tag is established in the vicinity of the food image on thebasis of a user input regarding the preset tag.

The display may be configured to display a list of foods stored in arefrigerating chamber on a refrigerating chamber management region, maydisplay the list of foods stored in a freezing chamber on a freezingchamber management region, and may display a list of foods stored in atemperature changeable chamber on a temperature changeable chambermanagement region.

The controller may be configured to generate a shopping list on thebasis of the generated food information, and may control the display todisplay the generated shopping list.

The refrigerator may further include a communication part configured tocommunicate with an external device.

The communication part may be configured to transmit the generated foodinformation to the external device upon receiving a request from theexternal device.

Advantageous Effects

As is apparent from the above description, the refrigerator according tothe embodiments of the present disclosure has the following effects.

First, the refrigerator can visualize functions thereof, and can providethe visualized functions to a user. Therefore, the user can easilyrecognize an indoor temperature of the storage chamber and a state offoods stored in the storage chamber.

The refrigerator includes a voice recognition function based on aproximity sensor so that the user can easily utilize necessary services.

The refrigerator can allow the user to easily manage foods stored in thestorage chamber through food input/output and management functions.

In addition, the refrigerator can communicate with the peripheral deviceover the network, resulting in high convenience of general services(e.g., TV services) frequently used by the user.

DESCRIPTION OF DRAWINGS

The accompanying drawings, which are included to provide a furtherunderstanding of the invention, illustrate embodiments of the inventionand together with the description serve to explain the principle of theinvention.

FIG. 1 is a perspective view illustrating a refrigerator according to anembodiment of the present disclosure.

FIG. 2 is a perspective view illustrating an internal structure of therefrigerator shown in FIG. 1.

FIGS. 3 and 4 are control block diagrams illustrating the refrigeratoraccording to an embodiment of the present disclosure.

FIG. 5 is a view illustrating an exemplary refrigerator managementscreen image.

FIGS. 6 to 8 are views illustrating exemplary management screen imagesof the corresponding region for use in a refrigerator map.

FIGS. 9 and 10 are views illustrating exemplary timer setting screenimages.

FIGS. 11 and 12 are views illustrating exemplary function explainingscreen images.

FIGS. 13, 14, 15, 16A, and 16B are views illustrating exemplaryenvironment setting screen images.

FIGS. 17, 18, 19, 20, 21, 22, and 23 are exemplary screen imagesvisually illustrating technology of the refrigerator.

FIG. 24 is a perspective view illustrating a refrigerator according toanother embodiment of the present disclosure.

FIGS. 25 and 26 are control block diagrams illustrating the refrigeratoraccording to another embodiment of the present disclosure.

FIG. 27 is a view illustrating exemplary formation of an input part of arefrigerator according to an embodiment of the present disclosure.

FIG. 28 is a conceptual diagram illustrating a voice recognition processof the refrigerator according to an embodiment of the presentdisclosure.

FIG. 29 is a conceptual diagram illustrating exemplary provision of avoice control screen image.

FIGS. 30 and 31 are exemplary views illustrating an activation processof the voice control screen image.

FIG. 32 is a view illustrating an example of visualizing a voicerecognition process and providing the visualized voice recognitionprocess in the voice control screen image.

FIG. 33 is a view illustrating an example of displaying the voicerecognition process in the voice control screen image.

FIGS. 34A and 34B are views illustrating examples of providing afunction execution screen image using a voice command

FIG. 35 is a view illustrating an example of providing a voice commandguide screen image.

FIG. 36 is a view illustrating a refrigerator according to anotherembodiment of the present disclosure.

FIGS. 37 and 38 are block diagrams illustrating the refrigeratoraccording to another embodiment of the present disclosure.

FIGS. 39 and 40 are views illustrating the principal screen images of afood reminder application according to an embodiment of the presentdisclosure.

FIG. 41 is a conceptual diagram illustrating a method for setting areminder tag by dragging a preset tag.

FIG. 42 is a conceptual diagram illustrating a method for adding a fooditem to a refrigerating chamber management region by clicking a food addbutton of the refrigerating chamber management region.

FIG. 43 is a view illustrating a refrigerator according to anotherembodiment of the present disclosure.

FIG. 44 is a block diagram illustrating the refrigerator according toanother embodiment of the present disclosure.

FIG. 45 is a block diagram illustrating the refrigerator according toanother embodiment of the present disclosure.

FIG. 46 is a view illustrating a device search screen image.

FIG. 47 is a view illustrating an exemplary channel search screen imagefor a display device.

FIG. 48 is a view illustrating an exemplary set-top box screen image ofa single channel

BEST MODE

Reference will now be made in detail to the embodiments of the presentdisclosure, examples of which are illustrated in the accompanyingdrawings, wherein like reference numerals refer to like elementsthroughout.

A refrigerator according to an embodiment may provide a refrigeratormanagement function, may visualize technical information thereof, andmay provide the visualized technical information.

FIG. 1 is a perspective view illustrating a refrigerator 1 according toan embodiment of the present disclosure. FIG. 2 is a perspective viewillustrating an internal structure of the refrigerator 1 shown in FIG.1.

Referring to FIGS. 1 and 2, the refrigerator 1 includes a main body 10,storage chambers (20, 30, 40) formed in the main body 10, and a cool airsupply device (not shown) to supply cool air to the storage chambers(20, 30, 40).

The main body 10 may include an inner casing to form the storagechambers (20, 30, 40); an outer casing coupled to the exterior of theinner casing so as to form the exterior appearance of the refrigerator1; and insulation disposed between the inner casing and the outer casingso as to insulate the storage chambers (20, 30, 40).

The storage chambers (20, 30, 40) may be classified into an upperstorage chamber 20 and lower storage chambers (30, 40) by a firstintermediate partition 21. In addition, the lower storage chambers (30,40) may be classified into a left storage chamber 30 and a right storagechamber 40 on the basis of the front surface of the refrigerator 1. Forconvenience of description, it is assumed that the upper storage chamber20 is used as a refrigerating chamber, the left lower storage chamber 30is used as a freezing chamber, and the right lower storage chamber 40 isused as a temperature changeable chamber.

The temperature of the refrigerating chamber 20 may be maintained atabout 3° C. to keep foods in a cooled state, while the temperature ofthe freezing chamber 30 may be maintained at about −18.5° C. to keepfoods in a frozen state. The temperature changeable chamber 40 may storefoods therein by changing a temperature according to a user desire.

The refrigerating chamber 20 may be provided with a shelf 23 allowingfood to be placed thereon, and at least one storage box 25 to store foodin a sealed state.

Also, an ice-making chamber 51 to make ice may be provided at an uppercorner of the refrigerating chamber 20, compartmentalized separatelyfrom the refrigerating chamber 20 by the ice-making chamber case 53. Theice-making chamber 51 may be provided with an ice-making tray to makeice, and an ice-making device 100 such as an ice bucket to store the icemade in the ice-making tray.

The refrigerating chamber 20 may be provided with a water tank 60 toretain water. The water tank 60 may be provided between storage boxes 25as shown in FIG. 2, but embodiments of the present invention are notlimited thereto. The water tank 60 may be placed at any position in therefrigerating chamber 20 so long as the position allows the water tank60 to be cooled by the cool air in the refrigerating chamber 20.

The refrigerating chamber 20 and the freezing chamber 30 mayrespectively have open front surfaces for entrance/exit of food. Theopen front surfaces of the refrigerating chamber 20 may be opened orclosed by doors (21, 22) hinged to the main body 10. The open frontsurfaces of the freezing chamber 30 and the temperature changeablechamber 40 may be opened or closed by the doors (31, 32) hinged to themain body 10. Door guards 24 for storing food therein may be provided atback surfaces of the refrigerating chamber doors (21, 22).

A gasket (not shown) for sealing a space between the main body 10 andthe refrigerating chamber doors (21, 22) when the refrigerating chamberdoors (21, 22) are closed may be provided to the rear borders of therefrigerating chamber doors (21, 22) so as to control cool air of therefrigerating chamber 20. In addition, a rotation bar (not shown) forlocking or unlocking cool air of the refrigerating chamber 20 by sealingthe space between the refrigerating chambers doors (21, 22) may beprovided to any one of the refrigerating chamber doors (21, 22).

In addition, a sensing part 70 for sensing opening or closing of thedoors of the refrigerator 1 may be provided to the rear borders of thestorage chamber doors (21, 22, 31, 32). Although the sensing part 70 isimplemented using a capacitive sensor, the scope or spirit of thesensing part 70 is not limited thereto.

In addition, one of the refrigerating chamber doors (21, 22) may beprovided with a dispenser 90 allowing a user to dispense water or ice atthe outside of the refrigerator without opening the refrigeratingchamber door (21, 22).

A control panel 100 may be mounted to one of the refrigerating chamberdoors (21, 22) such that the user located at the outside of therefrigerator can confirm a state of the refrigerator or can establishthe operation conditions of the refrigerator 1 without opening therefrigerating chamber doors (21, 22). The control panel 100 is mountedto a large display having a size of about 20 inches, resulting inincreased user convenience. However, the size of the control panel 100is not limited to examples of the above-mentioned numerical values.

A refrigerator management screen mage S1 may be displayed on the displayof the control panel 100. For example, when the doors of therefrigerator 1 are opened or closed, the display of the control panel100 may display a refrigerator map (M) or may display functions relatedto the indoor setting region. In addition, the indoor operation state,the indoor cold airflow, the food optimum for each indoor temperature,the operation state of the dispenser 25, and the indoor temperaturechange caused by the door opening/closing of the refrigerator 1 may bevisually displayed. Parts related to an exemplary display of the controlpanel 100 will hereinafter be described.

FIGS. 3 and 4 are control block diagrams illustrating the refrigerator 1according to an embodiment of the present disclosure.

Referring to FIGS. 3 and 4, the refrigerator 1 may include a controlpanel 100, a sensing part 70, a memory 75, and a controller 90.

The control panel 100 may provide a user interface (UI). Therefore, thecontrol panel 100 may be arranged at the front surfaces of therefrigerating chamber doors (21, 22) so that the user can easilyapproach and manipulate the control panel 100.

The control panel 100 may include the input part 110 for usermanipulation, and a display 140 for providing the user with necessaryinformation.

The input part 110 may receive a control command for the refrigerator 1from the user. The input part 110 may also be implemented as a hard keyfor user input, a proximity sensor, a Graphical User Interface (GUI)such as a touch pad. If the input part 110 is implemented as a GUI suchas a touch pad, the input part 110 is implemented as a touchscreenpanel, so that the input part 110 and the display 140 may construct adual-layer structure. For convenience of description, the followingdescription assumes that the input part 110 is implemented as atouchscreen panel.

The display 140 may provide the user with visual information indicatingthe indoor setting region. The display 140 may include a display 150 todisplay state information of the refrigerator 1 and a display 160 todisplay state information of the dispenser 25. For clarity of theabove-mentioned structure, the display 150 for displaying the state ofthe refrigerator 1 may be defined as a first display, and the display160 for displaying the state of the dispenser 25 may be defined as asecond display.

Although the first and second displays (150, 160) may display differentscreen images, a screen image of the first display 150 may also extendto the second display 160 as necessary. An example of the first display150 will hereinafter be described with reference to the associatedparts.

Each of the first and second displays (150, 160) may be implemented asany one of a Cathode Ray Tube (CRT), a Digital Light Processing (DLP)panel, a Plasma Display Panel, a Liquid Crystal Display (LCD) panel, anElectro Luminescence (EL) panel, an Electrophoretic Display (EPD) panel,an Electrochromic Display (ECD) panel, a Light Emitting Diode (LED)panel, and an Organic Light Emitting Diode (OLED) panel, without beinglimited thereto.

The sensing part 70 may collect the opening/closing information of thedoors of the refrigerator. The sensing part 70 may collect informationacquired when the user opens or closes the doors of the refrigerator 1.Although the sensing part 70 can collect the door opening/closinginformation of the refrigerator using the capacitive sensor or the like,the scope or spirit of the sensing part 70 is not limited thereto. Thedoor opening/closing information collected by the sensing part 70 may beapplied to the controller 90. The door opening/closing informationapplied to the controller 90 may be used as a trigger for displaying therefrigerator management screen image S1 of the first or second display150 or 160.

The memory 75 may store various data, control programs or applicationsneeded to drive or control the refrigerator 1. For example, the memory75 may control the refrigerator 1, and may visualize a control screenimage, such that it can store the refrigerator management program orapplication needed for visualization of the control screen image.

The memory 75 may store a User Interface (UI) related to the controlprogram or application needed to control the refrigerator 1, objects(e.g., images, text, icons, buttons, etc.) needed to provide the UI,user information, documents, databases (DBs), and/or related data. Forexample, the memory 75 may store a 2D or 3D refrigerator map (M) image,and may visualize images and text needed to visualize a temperature orstate of the storage chamber region as well as to provide the visualizedinformation to the user.

Upon receiving a control signal from the controller 90, the memory 75may store not only position information of a touch signal detected bythe input part 110, but also position information of successive touchsignals corresponding to successive touch movement.

The memory 75 may be implemented as at least one of a flash memory type,a hard disk type, a multimedia card micro card, a card type memory (e.g.a Secure Digital (SD) memory or an eXtreme Digital (XD) memory), aRandom Access Memory (RAM), a Static Random Access Memory (SRAM)), aRead Only Memory (ROM), an Electrically Erasable Programmable Read OnlyMemory (EEPROM), a Programmable Read Only Memory (PROM), a magneticmemory, a magnetic disk, an optical disc, etc., without being limitedthereto.

The controller 90 may control signal flow of the internal constituentelements of the refrigerator and overall operation of the refrigerator1, and may perform data processing. If a control command of the user isinput or a predetermined setting is satisfied, the controller 90 mayexecute the refrigerator management program or application stored in thememory 75.

The controller 90 may store the ROM 94 storing the control program orapplication for controlling the processor 92 and the refrigerator 1, andthe RAM 96 storing the external input signal or data of the refrigerator1 or used as a storage region corresponding to various tasks performedin the refrigerator 1. Hereinafter, the ROM 94 and RAM 96 of thecontroller 90 may conceptually include ROM and RAM of the memory 75.

The processor 92 may include a graphics processor for processing ofimages or video. The processor 92 may be implemented as an SoC (SystemOn Chip) including a Core and a GPU. In accordance with the embodiment,the controller 90 may include a graphics processor constructed in aseparate circuit board electrically connected to the controller 90, anda graphic processing board including ROM 94 or ROM 96.

If a user command is input or a predetermined condition is satisfied,the controller 90 may output first content corresponding to either theuser command or the predetermined condition to the first and seconddisplays (150, 160). If user touch is detected, the controller 90changes the first content corresponding to the detected user touch tothe synchronized second content, and outputs the second content to thefirst and second displays (150, 160).

The refrigerator management function of the refrigerator 1 according tothe embodiments will hereinafter be described with reference to theexample of the display 140 shown in FIGS. 5 to 16B.

Although FIGS. 5 to 16B are only examples of the screen imagedisplayable on the display 140, the scope of the present disclosure isnot limited to FIGS. 5 to 16B. Meanwhile, although the refrigeratormanagement screen image of FIGS. 5 to 16B can be displayed on the firstand second displays (150, 160) of the display 150, it is assumed thatthe refrigerator management screen image is provided to the firstdisplay 150 for convenience of description.

If the refrigerator management application is executed or the dooropening/closing information of the refrigerator 1 is received from thesensing part 70, the first display 150 may output the refrigeratormanagement screen image S1 shown in FIG. 5.

Referring to FIG. 5, words (e.g., fridge manager) indicating executionof the refrigerator management function may be displayed on a top bar(TB) of the refrigerator management screen image S1.

The refrigerator map (M) may be displayed at the center of therefrigerator management screen image S1, and the refrigerator map (M)may visualize and provide the refrigerator function information. Forexample, current temperature information of the corresponding region,setting temperature information or mode information may also bedisplayed on the refrigerator map (M). Although the refrigerator map (M)of FIG. 5 exemplarily illustrates that the storage chambers (20, 30, 40)of the refrigerator 1 are classified into the upper refrigeratingchamber 30, the left lower freezing chamber 30, and the right lowertemperature changeable chamber 40, the refrigerator map (M) may beprovided in different ways according to shapes of the refrigerator 1.

An information message for indicating functions of the refrigerator 1may be displayed at the lower end of the refrigerator management screenimage S1, various icons for operation control, function description, orenvironment setting of the refrigerator 1 may be displayed according tovarious embodiments. For example, the timer setting icon (I1), thefunction description icon (I2), and the environment setting icon (I3)may be displayed at the lower end of the information message.

The refrigerator management screen image S1 will hereinafter bedescribed in detail.

If the user touches the corresponding region of the refrigerator map (M)from among the refrigerator management screen S1, the first display 150may output the management screen image for the corresponding region inresponse to the user touch.

FIGS. 6 to 8 are views illustrating exemplary management screen imagesof the corresponding region for use in the refrigerator map (M).

If user touch is detected in the refrigerating chamber region of therefrigerator map (M), the refrigerating chamber management screen imageS2 may be displayed as shown in FIG. 6. Referring to FIG. 6, therefrigerating chamber expansion map M1 may be displayed at the center ofthe refrigerating chamber management screen image S2.

A refrigerating chamber function visualization region R1 may be providedto the refrigerating chamber expansion map M1 of the refrigeratingchamber management screen image S2. Images related to the refrigeratingchamber temperature, the cooling mode, etc. may be provided to therefrigerating chamber function visualization region R1. In addition, thecurrent temperature information of the refrigerating chamber 20 may bedisplayed on the refrigerating chamber expansion map M1. In accordancewith the embodiment, a setting information temperature of therefrigerating chamber 20 may be displayed on the refrigerating chamberexpansion map M1, or the icon indicating selection of a power-cool modemay also be displayed together with the setting information temperature.The power-cool mode is used to reduce the cooling time of foods storedin the storage chamber. After lapse of a predetermined time for energysaving, the power-cool mode may be automatically terminated.

The power cool mode setting icon (I4) and the temperature setting icon(I5) may be simultaneously displayed at the right side of therefrigerating chamber expansion map M1 of the refrigerating chambermanagement screen image S2. The user may rapidly cool the temperature ofthe refrigerating chamber 20 through the power-cool mode setting icon(I4), and may set the temperature of the refrigerating chamber 20 to adesired temperature through the temperature setting icon (I5).

The information message indicating functions of the refrigeratingchamber 20 may be displayed at the lower end of the refrigeratingchamber management screen image S2. Although the message “The idealtemperature setting for the refrigerator is 3° C.” may be displayed asan example, the scope or spirit of the displayable information messageis not limited thereto.

“Cancel” button B1 and “Save” button B2 may be displayed at thelowermost part of the refrigerating chamber management screen image S2.If the user touches the Cancel button B1, a cancel pop-up window may bedisplayed. If the user touches the Save button B2, user-input settinginformation may be stored.

Thereafter, if user touch is detected in the freezing chamber region ofthe refrigerator map (M), the freezing chamber management screen imageS3 may be provided as shown in FIG. 7. Referring to FIG. 7, therefrigerating chamber expansion map M1 may be displayed at the center ofthe freezing chamber management screen image S3.

A freezing chamber function visualization region R2 may be provided tothe freezing chamber expansion map M2 of the freezing chamber managementscreen image S3. Images related to the freezing chamber temperature, thecooling mode, etc. may be provided to the freezing chamber functionvisualization region R2. In addition, the current temperatureinformation of the freezing chamber 30 may be displayed on the freezingchamber expansion map M2. In accordance with the embodiment, a settinginformation temperature of the freezing chamber 30 may be displayed onthe freezing chamber expansion map M2, or the icon indicating selectionof a power-cool mode may also be displayed together with the settinginformation temperature.

The power-cool mode setting icon (I4) and the temperature setting icon(I5) may be simultaneously displayed at the right side of the freezingchamber expansion map M2 of the freezing chamber management screen imageS3.

An information message indicating functions of the freezing chamber 30may be displayed at the lower end of the freezing chamber managementscreen S3.

Although the message “The ideal temperature setting for a freezer is−19° C.” may be displayed as an example, the scope or spirit of thedisplayable information message is not limited thereto.

“Cancel” button B1 and “Save” button B2 may be displayed at thelowermost part of the freezing chamber management screen image S3.Redundant matters will not be described herein for clarity.

If the user touches the temperature changeable chamber region of therefrigerator map (M), the temperature changeable management screen imageS4 may be provided as shown in FIG. 8. The temperature changeablechamber expansion map M3 may be displayed at the center of thetemperature changeable chamber management screen image S4 as shown inFIG. 8.

The temperature changeable chamber function visualization region R3 maybe provided to the temperature changeable chamber expansion map M3 ofthe temperature changeable chamber management screen image S4.Temperature of the temperature changeable chamber 40, and images relatedto the temperature changeable mode may be provided to the temperaturechangeable chamber function visualization region M3. In addition, theicon indicating temperature changeable mode information established inthe temperature changeable chamber 40 may also be displayed on thetemperature changeable chamber expansion map M3.

Mode setting icons may be provided to the right side of the temperaturechangeable chamber expansion map M3 of the temperature changeablechamber management screen image S4. For example, the temperaturechangeable chamber mode may be classified into a freezer mode, a softfreezing mode, a chill mode, and a cool mode. In the freezer mode, theenvironment of the temperature changeable chamber 40 may be set to theenvironment of the freezing chamber 30. In the soft freezing mode, thetemperature of the temperature changeable chamber 40 is set to about −5°C., foods (e.g., fish, meat, etc.) may be stored in the refrigeratingchamber 20 for a long time. In addition, during the chill mode, thetemperature of the temperature changeable chamber 40 is set to about −1°C., and foods (e.g., fish, meat, etc.) may be stored in an optimum freshstate such that original quality of foods can be optimally stored in thetemperature changeable chamber 40. In the cool mode, the temperature ofthe temperature changeable chamber 40 is set to about 5° C., such thatbeverages, vegetables, and fruits may be optimally stored in thetemperature changeable chamber 40.

If the user touches one of the plurality of icons (i.e., the freezermode icon (I6), the soft freezing mode icon (I7), the chill mode icon(I8), and the cool mode icon (I9), the temperature changeable chamber 40may be set to the corresponding mode, information indicating that thecorresponding icon was selected in the vicinity of the correspondingmode setting icon (e.g., I8) may be provided.

The information message indicating the mode setting icon may bedisplayed at the lower end of the temperature changeable chambermanagement screen image S4. In the embodiment, not only the informationmessage, but also images indicating functions of the corresponding modesetting icon may be simultaneously displayed.

A Cancel button B1 and a Save button B3 may be displayed at thelowermost end of the temperature changeable chamber management screenimage S4. The same description as in FIG. 6 will herein be omitted forconvenience of description.

Referring back to FIG. 5, if the user touches the timer setting icon(I1) in the refrigerator management screen image 51, the first display150 may output the management screen image for the corresponding regionin response to the user touch.

FIGS. 9 and 10 are views illustrating exemplary timer setting screenimages.

If user touch (U1) is detected in the timer setting icon (I1) of therefrigerator management screen image S1, the timer setting screen imageS5 may be provided as shown in FIG. 9.

Referring to FIG. 9, words (e.g., Fridge Timer) indicating execution ofthe timer setting function may be displayed at the top bar (TB) of thetimer setting screen image S5. In accordance with the embodiment, theBack button B3 configured to return to the refrigerator managementscreen image S1 may be provided to the top bar (TB).

If user touch (U1) is input to the Back button B3, the refrigeratormanagement screen image S1 may be displayed on the first display 150.

The timer setting region R4 may be provided at the center of the timersetting screen image S5. The timer setting region R4 may include a firsttimer setting region (R4-1) and a second timer setting region (R4-2).The first timer setting region (R4-1) may include timers (T1: T1-1,T1-2, T1-3) initially provided to a manufacturer of the refrigerator 1,and the second timer setting region (R4-2) may include timers (T2: T2-1,T2-2) established by the user. If the timer T is in operation, anadditional display for indicating that the timer is in operation may beprovided to the operating timer. For example, the Stop button B5 and theCancel button B6, instead of the start button B4, may be provided to thetimer (T1-1).

The timer T1 of the first timer setting region (R4-1) may include thetimer name, the timer image, the set time, the set zone, and the startbutton B4.

The timer T2 of the second timer setting region (R4-2) may include thetimer name, the timer image, the set time, the set zone, the startbutton B4, and the delete button B7. The timer T2 of the second timersetting region (R4-2) may be a timer T2 randomly established by theuser. If user touch U1 is applied to the delete button B7, thecorresponding timer may be deleted. The same content as in theabove-mentioned description in association with other functions willherein be omitted for convenience of description.

The timer add timer B8 may be provided at the lower end of the timersetting screen image S5. If user touch U1 is applied to the timer addbutton B8, the timer add screen image S6 may be provided to the firstdisplay 150 as shown in FIG. 10. Words (e.g., Add Timer) indicatingexecution of the timer add function may also be displayed at the top bar(TB) of the timer add screen image S6.

A timer title bar (TTB) may be provided to the lower end of the top bar(TB). The title of the timer to be added may be established in the TTB,or the conventionally established time title may be corrected throughthe TTB. If the timer title is not established, a predetermined timertitle (e.g., timer01 or timer02) may be provided.

The time setting region R5 may be provided to the center of the timeradd screen image S6. The display region (R5-1) for displaying thesetting time may be provided to the time setting region R5, and thetimer picker button B9 for time setting may be provided in the vicinityof the display region.

The timer setting target designation region R6 may be provided at thelower end of the time setting region R5. If user touch U1 is input tothe timer setting target designation region R6, the list of timersetting possible regions may be provided. For example, although the listof timer setting possible regions including the refrigerating chamber20, the freezing chamber 30, and the temperature changeable chamber 40may be provided, and the scope or spirit of the list is not limitedthereto.

Cancel button B1 and Save button B2 may be provided at the lower end ofthe timer setting target designation region R6. The same description asin FIG. 6 will herein be omitted for convenience of description.

Referring back to FIG. 5, if user touch U1 is detected at the functiondescription icon (I2) in the refrigerator management screen image 1, thefirst display 150 may provide the function description screen image inresponse to the user touch U1.

FIGS. 11 and 12 are views illustrating exemplary function explainingscreen images.

If user touch U1 is detected in the function description icon (I2) ofthe refrigerator management screen image S1, the function descriptionscreen image S7 may be provided as shown in FIG. 11.

Referring to FIG. 11, words (e.g., “information”) indicating entry tothe function description screen image may be displayed at the top bar(TB) of the function description initial screen image S7. In accordancewith the embodiment, the Back button B3 configured to return to therefrigerator management screen image S1 may be provided to the top bar(TB).

The image region R7 for providing an image explaining functions may beprovided at the center of the function description screen image S7. Thetext region R8 for providing text explaining functions may be providedat the lower end of the function description screen image S7, and theicon list L1 regarding description provision functions may be providedat the lower end of the text region R8.

For example, the refrigerator map (M) may be displayed on the imageregion R7 of the function description screen image S7. The informationmessage for the user, for example, “Please select icon to viewinformation” may be provided to the text region R8. The icon list L1 mayinclude at least one of a “Precise Temperature Control” icon (I10), a“Maintain Optimum Food State” icon (I11), an “Energy Efficiency” icon(I12), and a “Purity Management” icon (I13). The scope or spirit ofexemplary icons contained in the icon list L1 is not limited thereto.Further, it should be noted that the present disclosure may be achievedin various ways through substitution, change, and modification by thoseskilled in the art without departing from the scope of the presentdisclosure as defined by the following claims.

If user touch U1 from among various icons provided to the icon list L1is received from the user, the detailed description screen related tothe corresponding function may be displayed.

For example, if the user touch U1 regarding the precise temperaturecontrol icon is received from the user as shown in FIG. 12, the functiondescription screen image S8 regarding precise temperature control may bedisplayed. Words (e.g., Precise Temperature Control) for displaying thefunction description target may be displayed at the top bar (TB) of thefunction description screen image S8.

In addition, the image region R7 for providing the function descriptionimage may be provided at the center of the function description screenimage S8, the text region R8 for providing text explaining functions maybe provided at the lower end of the function description screen imageS8, and the icon list L1 regarding the description providing function.The image indicator may also be displayed at the lower end of the imageregion R7 according to the embodiment. For example, the image forfunction description may be provided to the image region R7 of thefunction description screen image S8 regarding precise temperaturecontrol. If plural images are provided, the image indicator may also bedisplayed.

If the user touches (U1) the “Maintain Optimum Food State” icon, “EnergyEfficiency” icon, or “Purity Management” icon, the function descriptionscreen image may be provided as shown in FIG. 12.

Referring back to FIG. 5, if user touch U1 is detected in theenvironment setting icon (I3) from among the refrigerator managementscreen image S1, the first display 150 may provide the environmentsetting screen image in response to the user touch U1.

FIGS. 13, 14, 15, 16A, and 16B are views illustrating exemplaryenvironment setting screen images.

If the user touch U1 is detected in the environment setting icon (I3)from among the refrigerator management screen image S1, the environmentsetting screen image S9 may be provided as shown in FIG. 13. Referringto FIG. 13, words (e.g., Fridge Setting) indicating execution of theenvironment setting function may be displayed at the top bar (TB) of theenvironment setting screen image S9. In accordance with the embodiment,the Back button B3 configured to return to the refrigerator managementscreen image S1 may be provided to the top bar (TB).

The item list (CL) regarding the environment setting item C and thecorresponding items may be provided to the environment setting screenimage S9.

For example, the environment setting item C may include the refrigeratormanagement item (C-1) and the maintenance and user guidance item (C-2).

The refrigerator management item (C-1) of the environment setting item Cmay include environments regarding the refrigerator management function.For example, the refrigerator management item (C-1) may include the itemlist (CL1) comprised of the door alarm item (CI1) and the icemaker item(CI2). In addition, the on/off setting button (B10) constructed toswitch the corresponding item function on or off may also be provided atthe right side of the strings of the respective items (CI1, CI2).

The maintenance and user information item (C-2) from among theenvironment setting items (C) may provide user information regarding therefrigerator maintenance function or the like. For example, themaintenance and user information item (C-2) may provide the item list(CL2) including at least one of the energy saver item (CI3), the filteritem (CI4), the smart care item (CI5), and the demand reaction item(CI6). In accordance with the embodiment, the icon for visuallyproviding state information of the corresponding item may also beprovided at the right side of the string indicating each item. The icon(e.g., the icon for displaying a filter state using the color of thefilter icon) for visually providing state information of thecorresponding item may be provided at the right side of the stringdisplaying each item.

If user touch UI is detected in the items (CI1, CI2) of the refrigeratormanagement item (C-1), the detailed information screen image of thecorresponding function may be displayed. In accordance with theembodiment, if user touch U1 is detected in the on/off setting buttonB10 of the respective items (CI1, CI2), the corresponding function maybe established or released.

For example, if user touch U1 is detected in the door alarm item (CI1)as shown in FIG. 14, the detailed information screen image S10 for thedoor alarm function may be displayed.

Words (e.g., “Door Alarm”) indicating entry to the detailed informationscreen image of the door alarm function may be displayed at the top bar(TB) of the detailed information screen image S10. In accordance withthe embodiment, the Back button B3 configured to return to theenvironment setting screen image S9 may be provided to the top bar (TB).

The on/off setting bar (OFB) for switching the door alarm function on oroff may be provided at the lower end of the top bar (TB) of the detailedinformation screen image S10, and the on/off setting button B10 for thedoor alarm function may be provided at the on/off setting bar (OFB). Ifuser touch U1 is detected in the on/off setting button as shown in FIG.14, the on/off setting information for door alarm may be switched.

Meanwhile, the information region R9 for providing the text or imagesdescribing the door alarm function may be provided at the lower end ofthe on/off setting bar (OFB).

In accordance with the embodiment, if user touch U1 is detected in theon/off setting button of the door alarm item (CI1) as shown in FIG. 15,the on/off setting for the door alarm function may be switched. Forexample, assuming that the on/off setting button B10 for the door alarmfunction is switched off, if user touch U1 is detected in the on/offsetting button B10, the door alarm function may be switched on, and viceversa.

Thereafter, if user touch UI is detected in the items (CI3, CI4, CI5,CI6) of the maintenance and user information item (C-2), the detailedinformation screen image for the corresponding function may bedisplayed. In accordance with the embodiment, functions related torefrigerator maintenance may be switched on or off

For example, if user touch U1 is detected in the filter item (CI4) asshown in FIG. 16A, the detailed information screen image S11 of thefilter function may be displayed. The item for directing the filterfunction, the item for filter management, etc. may be provided to thedetailed information screen image S11 of the filter function. Forexample, the tutorial item (CI7) and the filter reset item (CI8) may beprovided to the detailed information screen image S11 of the filterfunction.

After the user receives the detailed description of the filter resetmethod by selecting the tutorial item (CI7), the user can reset thefilter by directly selecting the filter reset item (CI8).

If user touch U1 is detected in the on/off setting button B10 of thedemand reaction item (CI6) as shown in FIG. 16B, the on/off setting ofthe demand reaction function may be switched. The demand reactionfunction may change a power consumption pattern by reflecting economicalincentive and a signal of the electric charges in the demand reactionfunction, and the refrigerator 1 according to the embodiment may applythe demand response function related to the defrosting function.

For example, if the demand reaction item is switched on, therefrigerator 1 may receive information regarding the power consumptionfrom the electric power company. If the power consumption exceeds apredetermined reference power consumption on the basis of the powerconsumption received from the power company, the refrigerator 1 maycontrol operation of the refrigerator 1 in a manner that the function(e.g., the function of the defrosting operation) causing high powerconsumption of the refrigerator 1 is not performed.

The technical visualization function of the refrigerator 1 according tothe embodiment will hereinafter be described with reference to FIGS. 17to 23. Although FIGS. 17 to 23 are exemplary screen images visuallyillustrating technology of the refrigerator 1, the technical idea of thepresent invention is not limited thereto.

The refrigerator 1 according to the embodiment may visualize technologyrelated to the refrigerator management screen image S1 of FIGS. 5 to16B, and may provide the visualized technology to the display 140. Forconvenience of description, the embodiment of the present disclosurewill disclose the exemplary case in which technology related to thefirst display 150 of the display 140 is visualized.

The refrigerator 1 according to the embodiment may provide therefrigerator map (M) to the display (e.g., the first display 150)mounted to the refrigerator 1 when a predetermined condition issatisfied. The overall map of the refrigerator 1 may be provided to thefirst display 150 according to the situation, or the expansion map of aspecific storage chamber 20, 30 or 40 of the refrigerator 1 may beprovided. Current state information of the refrigerator 1 may beprovided to the overall map or the expansion map.

Referring to FIG. 17, assuming that the temperature changeable chamberdoor is opened for a predetermined first time on the basis ofinformation collected from the sensing part 70, the controller 90 of therefrigerator 1 may provide the expansion map M3 of the temperaturechangeable chamber 40 to the first display 150. In this case, the firsttime may be predetermined by the manufacturing company or the user. Thesetting temperature information of the temperature changeable chamber40, the current temperature information, etc. may be provided to theexpansion map M3 of the temperature changeable chamber 40 provided tothe first display 150. For example, the expansion map M3 of thetemperature changeable chamber 40 provided to the first display 150 mayinclude not only first content indicating that a current temperature ofthe temperature changeable chamber 40 is set to a temperature (e.g., 2°C.) appropriate for storage of fresh vegetables, but also second contentindicating that the doors of the temperature changeable chamber 40remain opened for a predetermined time so that the temperature of thetemperature changeable chamber 40 increases to 4° C.

As described above, the refrigerator 1 may provide the refrigerator map(M) (e.g., the expansion map of a specific region) to the first display150 on the basis of the refrigerator door opening/closing informationcollected by the sensing part 70. As a result, utilization of technologysupplied from the refrigerator 1 can be increased, and useraccessibility of the refrigerator 1 can be improved.

In addition, the refrigerator 1 according to the embodiment mayvisualize technology applied to the indoor setting region of the display(e.g., the first display 150) mounted to the refrigerator 1, and mayprovide the visualized technology to the first display 150. Inaccordance with the embodiment, the detailed description of the functioncan also be provided.

Referring to FIG. 18, the first display 150 may visualize the technologyapplied to the indoor setting region through the function descriptionscreen image S7, and may provide the visualized technology. Words (e.g.,Maintain Optimum Food State) for displaying the function descriptiontarget may be displayed at the top bar (TB) of the function descriptionscreen image S7.

The image region R7 for providing the function description image may beprovided at the center of the function description screen image S7, andthe text region R8 for providing text explaining the functions may beprovided at the lower end of the function description screen image S7.The icon list L1 regarding the functions may be provided at the lowerend of the function description screen image S7. The refrigerator 1according to the embodiment may visualize various images regarding theoptimum food state maintenance method and the principles thereof, andmay provide the visualized images to the image region R7, such thatuser's technical understanding and usability can be increased.

In addition, the refrigerator 1 according to the embodiment may3D-visualize the indoor operating technology of the refrigerator 1, andmay display the 3D-visualized technology on the display (e.g., the firstdisplay 150) mounted to the refrigerator 1.

Referring to FIG. 19, the first display 150 may display airflow ofindoor air as a 3D visual element, such that user satisfaction mayincrease when a specific function of the refrigerator 1 is used.

In addition, the refrigerator 1 according to the embodiment may displaythe setting temperatures of the respective indoor regions of eachchamber and the recommendation list of optimum foods of the respectivesetting temperatures on the display (e.g., the first display 150)mounted to the refrigerator 1. In the meantime, the refrigerator 1 maydisplay outdoor temperature and humidity, on/off states of the icemaker, etc. The refrigerator 1 may simultaneously display a guidemessage, a process message, etc. at the lower end of the 3D image.

Referring to FIG. 20, the first display 150 may activate the optimumfood recommendation icon (I14) for a temperature of a currenttemperature chamber 40 along with the temperature changeable chamberexpansion map M3, and may display the activated optimum foodrecommendation icon (I14) and the temperature food recommendation icon(I14), such that user satisfaction with the indoor setting temperaturemay increase.

The refrigerator according to the embodiment may interact with thedispenser 25, such that the refrigerator may visually displayinformation regarding the dispenser 25 on the display (e.g., the firstdisplay 150) interacting with the dispenser 25.

Referring to FIG. 21, the first display 150 may visualize the amount ofwater received from the dispenser 25, and display the visualizedinformation. Referring to FIG. 22, the first display 150 may visualizeand display the ice types (ice cubes, crushed ice, or the like) takenout of the ice maker. In accordance with the embodiment, the firstdisplay 150 may visualize and display the appearance of water coming outof the dispenser 25, such that user satisfaction when the user uses thedispenser 25 may increase.

In addition, the display of the refrigerator 1 according to theembodiment may visualize constant (or fixed) temperature technology whenthe indoor temperature increases by the opening or closing action of therefrigerator door. When a predetermined condition is satisfied inconjunction with the door alarm function, the embodiment may provide thealarm function for user recognition such that the user can quickly closethe refrigerator door, and may provide visual information indicatingthat the indoor temperature is maintained at a constant temperatureafter the refrigerator door is closed.

Referring to FIG. 23, assuming that the temperature changeable chamberdoor is opened on the basis of information collected by the sensing part70 and is then closed after lapse of a predetermined second time, thecontroller 90 of the refrigerator 1 may display the appearance ofrolling and decreasing of waves. The second time may be established bythe manufacturer or may be established by the user.

The refrigerator 1 may provide the first display 150 with visualinformation indicating the process in which the indoor temperature iskept at a constant temperature on the basis of the refrigerator dooropening/closing information collected by the sensing part 70, such thatuser satisfaction with the constant temperature technology applied tothe refrigerator 1 may increase.

The refrigerator 1 according to the embodiment has been described above.

A refrigerator according to another embodiment will hereinafter bedescribed in detail.

The refrigerator according to another embodiment may recognize uservoice on the basis of user access information collected by the proximitysensor, such that the refrigerator may allow the user to moreconveniently use the input scheme. In addition, the hard key along withthe display may be provided to the front surface of the refrigerator,such that the user can use various input methods.

FIG. 24 is a perspective view illustrating a refrigerator 1 a accordingto another embodiment of the present disclosure.

Referring to FIG. 24, the refrigerator 1 a includes a main body 10,storage chambers (20, 30, 40) formed in the main body 10, and a cool airsupply device (not shown) to supply cool air to the storage chambers(20, 30, 40).

The main body 10 may include an inner casing to form the storagechambers (20, 30, 40); an outer casing coupled to the exterior of theinner casing so as to form the exterior appearance of the refrigerator 1a; and insulation disposed between the inner casing and the outer casingso as to insulate the storage chambers (20, 30, 40).

The storage chambers (20, 30, 40) may be classified into an upperstorage chamber 20 and lower storage chambers (30, 40) by a firstintermediate partition (not shown). In addition, the lower storagechambers (30, 40) may be classified into a left freezing chamber 30 anda right temperature changeable chamber 40 on the basis of the frontsurface of the refrigerator 1 a. However, the partition example of thestorage chambers (20, 30, 40) is not limited thereto.

The storage chambers (20, 30, 40) may respectively have open frontsurfaces for entrance/exit of food. The open front surfaces of therefrigerating chamber 20 may be opened or closed by doors (21, 22)hinged to the main body 10. The open front surfaces of the freezingchamber 30 and the temperature changeable chamber 40 may be opened orclosed by the doors (31, 32) hinged to the main body 10.

In addition, the display 170 may be mounted to one of the refrigeratingchamber doors (21, 22) such that the user located at the outside of therefrigerator 1 a can confirm a state of the refrigerator 1 a or canestablish the operation condition of the refrigerator 1 a withoutopening the refrigerating chamber doors (21, 22). The button part 120formed in a hard key shape may be provided in the vicinity of thedisplay 170, such that it can provide the user with various inputmethods.

In addition, the sensing part 70 for detecting user access may beprovided to any one of the refrigerating chamber doors (21, 22), and thevoice input part 130 for collecting user voice information may also beprovided to any one of the refrigerating chamber doors (21, 22). Inaccordance with the embodiment, the camera provided for entrance of foodin the storage chambers (20, 30, 40) may be installed at any one of therefrigerating chambers (21, 22). The same description as in therefrigerator 1 of FIGS. 1 and 2 will herein be omitted for convenienceof description.

FIGS. 25 and 26 are control block diagrams illustrating the refrigerator(1 a) according to another embodiment of the present disclosure.

Referring to FIGS. 25 and 26, the refrigerator 1 a according to anotherembodiment may include a control panel 100, a sensing part 70, a memory75, and a controller 90.

The control panel 100 may be used to interface with the user. Therefore,the control panel 100 may be arranged at the front surfaces of therefrigerating chamber doors (21, 22) such that user access andmanipulation can be facilitated.

The control panel 100 may include an input part 110 for facilitatinguser manipulation, and a display 140 for providing information to theuser.

The input part 110 may receive a control command of the refrigerator 1 afrom the user.

This input part 110 may be implemented by at least one of variousschemes, for example, a hard key scheme, a proximity sensor scheme, anda GUI scheme such as a touchpad (U1). The touchpad U1 is implemented asa touchscreen panel, and the touchpad and the display 140 may constructa mutual layer structure.

The input part 110 according to the embodiment may be implemented as anyone of the above-mentioned input schemes so as to provide the user withvarious input schemes.

FIG. 27 is a view illustrating exemplary formation of the input part 110of the refrigerator (1 a) according to an embodiment of the presentdisclosure. Referring to FIG. 27, the input part 110 may include atouchscreen panel 112 incorporated with the display 140, a button part120 separated from the display 140, and a voice input part 130configured to collect voice information of the user.

The button part 120 may be provided at the front surfaces of therefrigerating chamber doors (21, 22). The button part 120 may beimplemented as a hard key or a touchscreen. The button part 120 mayoperate independently from the operation of the application providedfrom the refrigerator 1 a. That is, execution of the input commandgenerated by the button part 120 may have priority in certainsituations, such that the user can more easily use various functionssupplied from the respective buttons through the button part 120.

The button part 120 may be comprised of a combination of the pluralityof buttons. For example, the button part 120 may include a camera(CAMERA) button 120-1, a menu (MENU) button 120-2, a home (HOME) button120-3, a back button 120-4, and a notification (NOTI) button 120-5.However, the buttons are not limited thereto.

The voice input part 130 may collect control command informationprovided as a voice message from the user. Although the voice input part130 may include a microphone module to collect voice information of theuser, the scope of the voice input part 130 is not limited thereto. Thevoice input part 130 may convert the voice signal collected from theuser into a data format, and may provide the data format to the voicerecognition process of the controller 90.

The display 140 may visually provide the user with information regardingthe indoor setting region. The display 140 may include a display 170 fordisplaying state information of the refrigerator 1 a. In order todistinguish the display of this embodiment from the first and seconddisplays (150, 160) of the refrigerator 1 a, the display according tothe embodiment will hereinafter be defined as a third display 170.

The third display 170 may be constructed as a large display having asize of about 20 inches. However, the size of the third display 170 isnot limited thereto. In addition, the third display 170 may beconfigured to receive user touch input. In contrast, in association withthe category of the third display 170, the same description as in thefirst and second displays (150, 160) will herein be omitted forconvenience of description.

The sensing part 70 may collect user access information contained in thesensing region. Although the sensing part 70 may include the proximitysensor 71, the scope or spirit of the sensing part 70 is not limitedthereto, and all constructions for collecting access of the user locatedin the vicinity of the refrigerator 1 a may be used. For example, thesensing part 70 is implemented as the proximity sensor 71 forconvenience of description.

The proximity sensor 71 may be installed in the refrigerating chamberdoors (21, 22). The proximity sensor 71 may always operate wheninformation collected by the proximity sensor 71 is needed when aspecific program or application operates in the refrigerator 1 a, andthe collected information may be provided to the control process of thecontroller 90 upon receiving a request from the controller 90.

The memory 75 may store various data for driving and controlling therefrigerator 1 a, a control program or application therein. In moredetail, the memory 75 may perform user voice recognition, and may storedata, a control program or an application needed to control therefrigerator 1 a. In accordance with the embodiment, the memory 75 maystore the application related to user voice recognition (e.g.,Food-Reminder application, Morning-Brief application, etc.) therein, andthese applications may be stored in the initial stage, or may bedownloaded from the user during operation of the refrigerator 1 a.

The memory 75 may store a control program to control the refrigerator 1a, User Interface (UI) related to the application, an object (e.g.,image, text, icon, button, etc.) to provide the UI, user information,documents, databases (DBs), or associated data. For example, the memory75 may display user information to identify user voice, may display useraccess, and may store an object for displaying a user voice recognitionstate.

Upon receiving a control signal from the controller 90, the memory 75may store a touch signal U1 detected by the input part 110, and maystore position information of successive touch signals U1 correspondingto successive movement of the touch action U1. In addition, the memory75 may store input signal information detected by the button part 120.

The controller 90 may control flow of signals between internalconstituent elements of the refrigerator 1 a and overall operation ofthe refrigerator 1 a, and may process data. The controller 90 mayperform the control program or application stored in the memory 75 whena control command (e.g., input of the button part 120, a voice command,etc.) from the user is input or a predetermined condition (e.g., whenthe proximity sensor 71 detects user access) is satisfied.

The controller 90 may include a Read Only Memory (ROM) 94 for storingthe control program or application needed to control the processor 92and the refrigerator 1 a; and a Random Access Memory (RAM) 96 forstoring the external input signal or data of the refrigerator 1 a orused as a storage region corresponding to various tasks executed in therefrigerator 1 a. ROM 94 and RAM 96 of the controller 90 mayconceptually include ROM and RAM of the memory 75.

The processor 92 may include a graphics processor to process images orvideo data. The processor may be implemented as a System on Chip (SoC)including a Core and a GPU. In accordance with the embodiment, thecontroller 90 may include a graphics processor and a graphics processingboard including RAM or ROM in a separate circuit board electricallyconnected to the controller 90.

The processor 92 may include a voice recognition processor forprocessing and interpreting a user voice signal. In accordance with theembodiment, the controller 90 may include a voice recognition processorand a voice recognition processing board including ROM or ROM in aseparate circuit board electrically connected to the controller 90. Thecontroller 90 may perform voice recognition of an isolated languagethrough the voice recognition processor, and may provide the user with aSpeech to Text (STT) regarding a voice command.

The embodiments of the present disclosure will hereinafter be describedwith reference to the attached drawings.

FIG. 28 is a conceptual diagram illustrating a voice recognition processof the refrigerator (1 a) according to an embodiment of the presentdisclosure.

Referring to FIG. 28, if the user inputs a voice recognition activationcommand to the refrigerator 1 a, or if a predetermined condition issatisfied, the refrigerator 1 a may transition from a voice recognitiondeactivation state 200 to a voice recognition activation state 300.

If the user inputs a voice command to the refrigerator 1 a in the voicerecognition activation state 300, the voice recognition process iscarried out. The voice recognition process may be carried out afterpassing through the voice recognition preparation step 310, the voiceinput step 320, the voice processing step 330, the result provision step340, and the result provision completion step 350. The voice recognitionprocess of the present disclosure may recognize isolated language andnatural language, and the following user interface (UI) provision methodcan be applied to both the isolated language recognition process and thenatural language recognition process.

The third display 170 may provide text, icons, or animation screenimages regarding the corresponding steps according to progression of thevoice recognition process, and may also provide an Auditory UserInterface (AUI) according to the embodiment.

In the result provision completion step 350, if the voice recognitionactivation command from the user is received, or if the predeterminedcondition is satisfied, the voice recognition process may be carried outagain. In addition, if the voice recognition deactivation command fromthe user is received, or if the predetermined second time elapses, therefrigerator 1 a may switch to the voice recognition deactivation state200.

The process for switching to the voice recognition activation state 300in the voice recognition deactivation state 200 will hereinafter bedescribed.

If the voice recognition activation command from the button part 120 isinput, or if the proximity sensor 71 detects user access, the controller90 may provide a voice control screen image to the third display 170. Inaddition, if the predetermined voice signal is recognized, thecontroller 90 may also provide the voice control screen image to thethird display 170. Subsequently, if the user voice command is inputthrough the voice input part 130, the voice recognition processregarding the input voice command may be visually displayed on the voicecontrol screen image of the third display 170.

FIG. 29 is a conceptual diagram illustrating exemplary provision of avoice control screen image. FIGS. 30 and 31 are exemplary viewsillustrating an activation process of the voice control screen image.

Referring to FIG. 29, the voice control screen image S12 may be providedat the top of the third display 170. The voice control screen image S12may include an environment setting icon (I15), a voice recognition statedisplay region R10, a help icon (I16), and a close icon (I17).

If user touch U1 is input to the environment setting icon (I15), theenvironment setting screen image may be displayed. If user touch U1 isinput to the help icon (I16), the help popup menu may be displayed. Ifuser touch U1 is input to the close icon (I17), the voice control screenimage S12 may be closed and the voice recognition function may also beended.

The voice recognition state display region R10 may display the voicerecognition process using text, images, or animation images, such thatthe user can easily recognize the voice recognition state. The voicecontrol screen image S23 may be activated as follows.

Referring to FIG. 30, if the user touches (U1) the notification (NOTI)button 120-5 of the button part 120, the home screen image S13 of thethird display 170 may be switched to the notification screen image S14.If the user touches (U1) the microphone icon (I18) provided at the topof the notification screen image S14, the notification screen image S14may be switched again to the home screen image S13. The voice controlscreen image S12 may also be provided at the top of the home screenimage S13. However, the scope of the method for providing the voicecontrol screen image S12 is not limited thereto. Of course, the voicecontrol screen image S12 may also be provided at the lower end of thehome screen image. In addition, the method for activating the voicecontrol screen image S12 may also be achieved by user voice recognitionor the like as described above.

Referring to FIG. 31, when the third display 150 is turned off or whenthe screen saver is displayed on the third display 170, if the proximitysensor 71 detects user access, the controller 90 may provide the thirddisplay 170 with the voice control screen image S12.

Although the third display 170 may provide not only the latest displayscreen image but also the voice control screen image S12 provided at thetop of the latest display screen image, the method for providing thevoice control screen image S12 is not limited thereto, and the voicecontrol screen image S12 may also be provided at the lower end of thelatest displayed screen image. If the voice control screen image S12 isactivated, the voice command from the user may be input. Upon receivingthe voice command from the user, the controller 90 may visualize thevoice recognition process on the voice recognition state display regionR10.

FIGS. 32 and 33 are views illustrating exemplary display of the voicerecognition process in the voice control screen image S12.

Referring to FIG. 32, if the voice control screen image S12 isactivated, the animation image indicating the voice recognitionpreparation state may be displayed on the voice recognition statedisplay region R10.

If the voice command from the user is input, the animation imageindicating that the voice command from the user is currently input maybe displayed on the voice recognition state display region R10.

If the voice command from the user is input, the image indicating thatvoice processing is currently executed may be displayed on the voicerecognition state display region R10.

If voice processing is completed, the voice recognition result may beprovided as text on the voice recognition state display region R10.

After the voice recognition result is provided, if a predetermined thirdtime elapses or an AUI for the text is provided, the icon indicatingcompletion of the voice recognition process may be displayed on thevoice recognition state display region R10.

Meanwhile, if voice recognition is not performed in the voicerecognition process, the voice recognition state display region R10 maynot provide the text-shaped voice recognition result.

Referring to FIG. 33, if voice recognition is not performed in the voicerecognition process, the image indicating the current voice processingstate is displayed on the voice recognition state display region R10,and the icon indicating completion of the voice recognition process mayalso be displayed thereon.

The displaying scheme of the voice recognition state display region R10is not limited to the examples of FIGS. 32 and 33, and the voicerecognition process may also be provided according to other schemesother than the above-mentioned schemes.

If a voice command from the user is input to the controller 90, thecontroller may perform the refrigerator (1 a) function corresponding tothe voice command, and may provide the function execution screen imagecorresponding to the corresponding function to the third display 170.

FIGS. 34A and 34B are views illustrating examples for providing afunction execution screen image S15 using a voice command

Referring to FIGS. 34A and 34B, if a voice command of the user is input,the voice recognition process may be visualized and displayed on thevoice recognition state display region R10 of the voice control screenimage S12.

For example, if the user inputs a voice command “Open Memo”, the thirddisplay 170 may be switched to the refrigerator function executionscreen image S15 corresponding to the “Open Memo” command That is, thememo program or the memo application of the refrigerator 1 a may beexecuted according to a user voice command, and the screen image of thethird display 170 may be switched to the home screen image regarding thememo function.

If successive voice commands from the user are input, the controller 90may execute the corresponding function of the refrigerator 1 a inresponse to the successive voice commands, and may display the executionscreen image S15 of the corresponding function on the third display 170.

For example, assuming that the user desires to add the plurality ofitems to the shopping list of the food reminder application, if the userinputs the command “Add Item”, the screen image of the third display 170may be switched to the image of the shopping list contained in the foodreminder application. Thereafter, if the user further inputs thecommands (e.g., “Apple”, “Orange”, and “Water”), the apple, orange, andwater may be added to the shopping list. In other words, the user mayadd the plurality of items to the shopping list by successivelyinputting the food names

If the user touches (U1) the help icon of the voice control screen imageS12, the controller 90 may provide the voice command guide screen image.

FIG. 35 is a view illustrating an example for providing a voice commandguide screen image.

Referring to FIG. 35, if the user touches (U1) the help icon (I16) ofthe voice control screen image S12, the voice command guide screen imageS16 may be provided. If the words (e.g., “Voice Control Function”)indicating the entry to the voice command guide screen image may bedisplayed at the top of the voice command guide screen image S16, andthe voice control function list L2 may be provided at the lower end ofthe voice command guide screen image S16. The voice control functionlist L2 may include the plurality of voice control items (LI1, LI2, LI3,LI4, LI5, LI6), and each item may include a title part and expansiondisplay button B11. Description of the corresponding items may bedisplayed at the lower end of the title part, and the latest commandsfor the corresponding items may be displayed at the lower end of thetitle part. In addition, if user touch U1 is input to the expansiondisplay button B11, all commands for the corresponding item (LI4) may bedisplayed.

The refrigerator 1 a according to the embodiment has been describedabove.

The refrigerator configured to provide the food entrance/exit managementfunction according to another embodiment will hereinafter be described.

In accordance with one aspect of the present disclosure, therefrigerator has the food capture function, and can manage foods storedin the compartment on the basis of the captured information. Morespecifically, the refrigerator or the mobile terminal 500 can managefoods according to the captured information.

In accordance with another aspect of the present disclosure, therefrigerator can receive the purchase items of the shopping list inassociation with the shopping list, and can manage the stored foods onthe basis of the purchase information.

In accordance with another aspect of the present disclosure, therefrigerator can receive the purchase list of the user in associationwith the shopping mall, and can manage the stored foods on the basis ofthe purchase information.

FIG. 36 is a view illustrating the refrigerator 1 b according to anotherembodiment of the present disclosure. FIGS. 37 and 38 are block diagramsillustrating the refrigerator 1 b according to another embodiment of thepresent disclosure.

Referring to FIGS. 36 to 38, the refrigerator 1 b according to anotherembodiment may exchange data with the mobile terminal 500 through theserver 400.

The refrigerator 1 b according to another embodiment may include a mainbody 10, storage chambers (20, 30, 40) formed in the main body 10, and acool air supply device (not shown) to supply cool air to the storagechambers (20, 30, 40).

The main body 10 may include an inner casing to form the storagechambers (20, 30, 40); an outer casing coupled to the exterior of theinner casing so as to form the exterior appearance of the refrigerator 1b; and insulation disposed between the inner casing and the outer casingso as to insulate the storage chambers (20, 30, 40).

The storage chambers (20, 30, 40) may respectively have open frontsurfaces for entrance/exit of food. The open front surfaces of therefrigerating chamber 20 may be opened or closed by refrigeratingchamber doors (21, 22) hinged to the main body 10. The open frontsurfaces of the freezing chamber 30 and the temperature changeablechamber 40 may be opened or closed by the doors (31, 32) hinged to themain body 10.

The display 180 may be mounted to one of the refrigerating chamber doors(21, 22) such that the user located at the outside of the refrigerator 1b can confirm a state of the refrigerator 1 b or can establish theoperation condition of the refrigerator 1 b without opening therefrigerating chamber doors (21, 22).

In addition, one of the refrigerating chamber doors (21, 22) may includethe capturing part 80 configured to capture bar codes attached to thestored foods or a packing container of the foods.

The server 400 may be connected to the Internet, and may store userinformation of the refrigerator 1 b and the mobile terminal 500. Themobile terminal 500 may communicate with the refrigerator 1 b over thenetwork. The mobile terminal 500 may include various applicationsrelated to remote control, monitoring, or food management functions.Thus, the user can confirm a state of the stored foods or manageentrance/exit of the foods using the mobile terminal 500. Therefrigerator 1 a will hereinafter be described in detail.

The refrigerator 1 b may include a control panel 100, a capturing part80, a communication part 86, a memory 75, and a controller 90.

The control panel 100 may interface with the user. Therefore, thecontrol panel 100 may be arranged at the front surfaces of therefrigerating chamber doors (21, 22) so that the user can easilyapproach and manipulate the control panel 100.

The control panel 100 may include the input part 110 for usermanipulation, and a display 140 for providing the user with necessaryinformation.

The input part 110 may receive a control command for the refrigerator 1b from the user, and the display 140 may display the screen image formanaging foods. The input part 110 and the display 140 may beincorporated with each other. More particularly, the display 140 mayinclude the display 180 mounted to the front surfaces of therefrigerating chamber doors (21, 22), and the input part 110 may includea touchscreen panel 112 incorporated with the display 180 provided tothe front surfaces of the refrigerating chamber doors (21, 22). In orderto distinguish the display 180 from the first to third displays (150,160, 170), the display 180 will hereinafter be referred to as the fourthdisplay 180 according to the embodiment.

The capturing part 80 may include a camera configured to capture barcodes attached to the stored foods or a packing container of the foods.The capturing part 80 may include a first capturing part 82 to capturefoods stored in the chamber, and a second capturing part 84 to capturebar codes attached to the stored foods or a packing container of thefoods.

The first capturing part 82 may be installed in the chamber. Moreparticularly, the first capturing part 82 may be installed in the Frenchbar of each of the storage chamber doors (21, 22, 31, 32), and thenumber of the first capturing parts 82 may be identical to the number ofshelves contained in each storage chamber (20, 30, 40). Although thefirst capturing part 82 can be installed in each storage chamber (20,30, 40), the following description assumes that the first capturing part82 is installed in the refrigerating chamber 20 for convenience ofdescription.

The first capturing part 82 may capture the indoor state of therefrigerating chamber as soon as the refrigerating chamber doors (21,22) are opened and closed. In accordance with the embodiment, thespecific time at which the refrigerating chamber doors (21, 22) areopened and closed may be detected by a circuit mounted to therefrigerating chamber doors (21, 22) or the French bar, and the firstcapturing part 82 may capture the indoor state by this detection signal.

A circuit, lens, tool, or software of the camera contained in the firstcapturing part 82 may be configured in a manner that a viewing angle ofthe camera can maintain a predetermined first angle or higher. Inaddition, the camera of the first capturing part 82 may include a heaterlocated in the vicinity of the camera lens or the circuit, and theheater may be controlled in a manner that the ambient temperature of thecamera is maintained at a predetermined temperature or higher.Therefore, the influence that humidity or dew condensation of the indoorspace of the refrigerating chamber 20 is applied to the lens or circuitof the camera can be minimized.

The second capturing part 84 may be installed in the storage chamberdoors (21, 22, 31, 32). The second capturing part 84 may be installed inany of the storage chamber doors (21, 22, 31, 32), and may be installedat the front surface of any one of the refrigerating chamber doors (21,22) so that the user can easily approach the refrigerating chamber doors(21, 22).

The second capturing part 84 may capture bar codes attached to thestored foods or a packing container of the foods.

The communication part 86 may control the refrigerator 1 b tocommunicate with the external device under the control of the controller90. The communication part 86 may receive a remote control signal fromthe mobile terminal 500 under the control of the controller 90, suchthat the communication part 86 can connect the refrigerator 1 b to themobile terminal 500.

The communication part 86 may include a wireless Ethernet, a wirelessLAN (WLAN), and a local area network (LAN) unit according to performanceand structure of the refrigerator 1 b.

The LAN unit may be wirelessly connected to an Access Point (AP) at anAP installed position according to a control signal of the controller90. The LAN unit may support IEEE1002.11x. Short-range communication ofthe LAN unit may include Bluetooth, Bluetooth low energy, infrared dataassociation (IrDA), Wireless Fidelity (Wi-Fi), Ultra Wideband (UWB),Near Field Communication (NFC), etc.

The memory 75 may store various data, control programs or applicationsto drive and control the refrigerator 1 b. More particularly, the memory75 may store data, control program, and applications to manage foodsstored in the refrigerator 1 b. For example, the memory 75 may store theapplication (e.g., Food-Reminder application, etc.) related to foodmanagement of the refrigerator 1 b, and the application may be stored inthe initial stage, or may be downloaded from the user during operationof the refrigerator 1 b.

The memory 75 may store a control program to perform food management,User Interface (UI) related to the application, an object (e.g., image,text, icon, button, etc.) to provide the UI, user information,documents, DBs, or associated data. For example, the memory 75 may storefood information to identify bar codes attached to the packing containerof the food, food image information to identify the food captured by thecapturing part 80, food image information to display an image of thefood captured by the capturing part 80 on the display 140, and images ortext information to allow the food reminder application to manage theexpiration date of the stored foods.

The controller 90 may control signal flow of the internal constituentelements of the refrigerator 1 b and overall operation of therefrigerator 1 b, and may perform data processing.

If the user inputs a control command or a predetermined condition issatisfied, the controller 90 may perform the control program orapplication stored in the memory 75.

The controller 90 may include a Read Only Memory (ROM) 94 for storingthe control program or application needed to control the processor 92and the refrigerator 1 b; and a Random Access Memory (RAM) 96 forstoring the external input signal or data of the refrigerator 1 b orused as a storage region corresponding to various tasks executed in therefrigerator 1 b. ROM 94 and RAM 96 of the controller 90 mayconceptually include ROM and RAM of the memory 75.

The processor 92 may include a graphics processor to process images orbar codes. The processor 92 may be implemented as a System on Chip (SoC)including a Core and a GPU. In accordance with the embodiment, thecontroller 90 may include a graphics processor and a graphics processingboard including RAM or ROM in a separate circuit board electricallyconnected to the controller 90.

The embodiment of the present disclosure will hereinafter be describedwith reference to the attached drawings.

The refrigerator 1 b according to the embodiment may provide the userwith the stored food management function using the food reminderapplication. The food reminder application may provide the user withvisual information indicating the presence or absence of the storedfoods, the expiration date of the foods, etc. The user may recognize theabove-mentioned information through the display 140 (e.g., the fourthdisplay 180) of the refrigerator 1 b or the display screen of the mobileterminal 500.

FIGS. 39 and 40 are views illustrating the principal screen images of afood reminder application according to an embodiment of the presentdisclosure.

FIG. 39 is a view illustrating the main screen image S17 of the foodreminder application, and FIG. 40 is a view illustrating the shoppinglist transmission screen image S180.

Referring to FIG. 39, the main screen image S17 of the food reminderapplication may include a preset tag region R10, a food display regionR11, a refrigerating chamber management region R12, a freezing chambermanagement region R13, and a temperature changeable chamber managementregion R14. The preset tag region R10 may be provided at the top of themain screen S17, and the food display region R11 may be installed at thecenter of the main screen image S17. The refrigerating chambermanagement region R12, the freezing chamber management region R13, andthe temperature changeable management region R14 may be provided at thelower end of the food display region R11.

The plurality of preset tags (PT) to which the reminder tag (RT) can beeasily attached may be mounted to the food image (I20) provided to thefood display region R11. In addition, a calendar icon (I19) to which thereminder tag (RT) is attached may be provided at the food image usingthe calendar.

The visual image of the refrigerating chamber 20, the images (I20) offoods stored in the refrigerating chamber 20, and the reminder tags (RT)attached to the respective food images (I20) may be displayed on thefood display region (R11). The visual image of the refrigerating chamber20 may be provided from the camera of the first capturing part 82mounted to the refrigerating chamber. Of course, the visual image of therefrigerating chamber 20 may also be provided from the camera installedat a rotation bar.

Meanwhile, the food image (I20) of the food display region R11 may beperiodically updated. More particularly, the controller 90 may controlthe camera of the first capturing part 82 to capture the foods stored inthe refrigerating chamber 20 using the opening/closing information ofthe refrigerating chamber doors (21, 22) as a trigger. The controller 90may receive capture information from the first capturing part 82, andmay update the food image (I20) displayed on the food display regionR11.

The reminder tag (RT) may provide the user with food information (e.g.,the expiration date information). The reminder tag (RT) may be displayedin different ways according to information to be provided by thereminder tag (RT). For example, the reminder tag (RT) indicating thatthe remaining time of the expiration dates is at least 7 days may bedenoted by RT1. The reminder tag (RT) indicating that the remaining timeof the expiration dates is shorter than 7 days may be denoted by RT2.

The refrigerating chamber management region R12 may display the list offoods stored in the refrigerating chamber 20. The food items of the foodlist may be added by clicking the food add button B12 contained in therefrigerating chamber management region R12. In accordance with theembodiment, the foods included in the door guard of the refrigeratingchamber 20 may not be displayed on the food display region R11, and thefoods may be managed through the refrigerating chamber management regionR12.

The freezing chamber management region R13 may display the list of foodsstored in the freezing chamber 30. The food items of the food list maybe added by clicking the food add button B12 contained in the freezingchamber management region R13. In accordance with the embodiment, thefoods stored in the refrigerating chamber 20 may be mainly displayed onthe food display region R11, such that the foods stored in the freezingchamber 30 may be managed through the freezing chamber management regionR13.

The temperature changeable chamber management region R14 may display thelist of foods stored in the temperature changeable chamber 40. The fooditems of the food list may be added by clicking the food add button B12contained in the temperature changeable chamber management region R14.In accordance with the embodiment, the foods stored in the refrigeratingchamber 20 may be mainly displayed on the food display region R11, suchthat the foods stored in the temperature changeable chamber 40 may bemanaged through the freezing chamber management region R13.

In accordance with the embodiment, the button “Send to Shopping List”B13 may be provided at the lower end of the main screen image S17. Ifthe user touches (U1) the “Send to Shopping List” button B13, theshopping list send screen image S18 for selecting whether to send thefood list contained in the food reminder application to the shoppinglist application may be provided.

Referring to FIG. 40, the “All select” button B14, the “Back” button B3,the “Cancel” button B1, the “Finish” button B15, etc. may be provided atthe top bar (TB) of the shopping list transmission screen image S18. Theshopping list transmission item C1 and the item lists (CL3, CL4, CL5)for the corresponding items may be provided at the lower end of the topbar (TB).

The shopping list transmission item C1 may include the “Recently RemovedItems” item C1-1, the refrigerating chamber food transmission item(Fridge) C1-2, and the freezing chamber food transmission item (Freezer)C1-3, and the item lists (CL3, CL4, CL5) for the corresponding items maybe displayed on the respective items.

The food management function of the refrigerator 1 b according to theembodiment will hereinafter be described with reference to the attacheddrawings.

For example, the reminder tag (RT) may be established by dragging thepreset tag (PT) to the corresponding food image of the food displayregion R11.

FIG. 41 is a conceptual diagram illustrating a method for setting areminder tag (RT) by dragging a preset tag (PT).

Referring to FIG. 41, if the user drag touch U2 is input to the presettag (PT), the reminder tag (RT) may be created at the finish time of theuser drag touch U2. In accordance with the embodiment, the reminder tag(RT) may be dragged again so that it can be movable. If the user touchU1 is input to the reminder tag (RT), the popup menu for editing thereminder tag (RT) may be provided.

In accordance with the embodiment, the user may drag the calendar icon(I19) to the corresponding food image of the food display region R11 sothat the reminder tag (RT) may be established. The user may establishthe reminder tag (RT) by dragging the calendar icon (I19), so that aschedule not provided from the preset tag (PT) may be established.

In addition, the user may add the food item to the refrigerating chambermanagement region R12 by clicking the food add button B12 of therefrigerating chamber management region R12. Foods may also be added tothe freezing chamber management region R13 and the temperaturechangeable chamber management region R14 in the same manner as in therefrigerating chamber management region R12, and detailed description ofthe refrigerating chamber management region R12 may include detaileddescription of the freezing chamber management region R13 and thetemperature changeable chamber management region R14.

FIG. 42 is a conceptual diagram illustrating a method for adding a fooditem B12 to a refrigerating chamber management region R12 by clicking afood add button of the refrigerating chamber management region R12.

Referring to FIG. 42, if user touch U1 is input to the food add buttonB12 of the refrigerating chamber management region R12, the food addpopup menu P1 may be provided. The food add popup menu P1 may includethe test input region R16, the reminder schedule setting region S17, andthe image region R18.

The user may directly input the names of foods desired to be managedthrough the food add popup menu P1, and may also input the reminderdate, the food image, etc. through the food add popup menu P1. The foodimage may provide the image of food directly captured by the camera ofthe second capturing part 84. In accordance with the embodiment, thefood image acquired through bar codes of the food container or the imageof a photo album may be provided.

Meanwhile, through account interworking the contracted shopping mall andthe food reminder application, the user may view or inquire about thelist of foods purchased from the shopping mall through the food reminderapplication, and it may be possible to generate and manage the foodinformation on the basis of the corresponding data.

In addition, data of the food reminder application is shared with dataof the shopping list application in a manner that food information canalso be created and managed. That is, the food reminder application mayreceive the food purchase list from the shopping list application, andmay automatically update the overall list to the food reminderapplication, such that food information can be created and managed.

In contrast, the user may view the food list through the shopping listtransmission screen image S18 of the food reminder application, and maytransmit necessary foods to the shopping list application such that thecorresponding food list may also be added to the memo list of theshopping list application.

In addition, the refrigerator 1 b may receive indoor image informationof each chamber from the mobile terminal 500, may generate foodinformation, and may manage the generated food information. In thiscase, the indoor image information may be captured by the mobileterminal 500.

In addition, the refrigerator 1 b may provide food managementinformation to the mobile terminal 500 upon receiving a request from themobile terminal 500. In accordance with the embodiment, the screen imageof the fourth display 180 of the refrigerator 1 b may be mirrored on thedisplay 140 of the mobile terminal 500, such that the user can recognizeinformation of the stored foods through the mobile terminal 500 while inmotion.

The refrigerator 1 b according to the embodiment has been describedabove.

The refrigerator for providing images mirrored on the display deviceaccording to another embodiment will hereinafter be described. In thiscase, the display device may include an image display device, forexample, television, desktop computer, laptop, smartphone, mobile phone,etc.

FIG. 43 is a view illustrating the refrigerator 1 c according to anotherembodiment of the present disclosure. FIG. 44 and FIG. 45 are a blockdiagram illustrating the refrigerator 1 c according to anotherembodiment of the present disclosure.

Referring to FIGS. 43 and 44, the refrigerator 1 c includes a main body10, storage chambers (20, 30, 40) formed in the main body 10, and a coolair supply device (not shown) to supply cool air to the storage chambers(20, 30, 40).

The main body 10 may include an inner casing to form the storagechambers (20, 30, 40); an outer casing coupled to the exterior of theinner casing so as to form the exterior appearance of the refrigerator 1a; and insulation disposed between the inner casing and the outer casingso as to insulate the storage chambers (20, 30, 40).

The storage chambers (20, 30, 40) may respectively have open frontsurfaces for entrance/exit of food. The open front surfaces of therefrigerating chamber 20 may be opened or closed by refrigeratingchamber doors (21, 22) hinged to the main body 10. The open frontsurfaces of the freezing chamber 30 and the temperature changeablechamber 40 may be opened or closed by the doors (31, 32) hinged to themain body 10.

The display 190 may be mounted to one of the refrigerating chamber doors(21, 22) such that the user located at the outside of the refrigeratorcan confirm a state of the refrigerator or can establish the operationcondition of the refrigerator without opening the refrigerating chamberdoors (21, 22).

The display 190 may communicate with the display device 600 over thenetwork. If the refrigerator 1 c is connected to the display device 600,the display 190 of the refrigerator 1 c may display the mirroring screenimage of the display device 600.

Detailed structures of the refrigerator 1 c and the display device 600will hereinafter be described.

The refrigerator 1 c may include a control panel 100, a communicationpart 86, a memory 75, and a controller 90. The display device 600 mayinclude a control panel 610, a communication part 620, a memory 630, anda controller 640.

In order to make a distinction between the refrigerator 1 c and thedisplay device 600, the control panel 100 of the refrigerator 1 c willhereinafter be referred to as a first control panel 100, thecommunication part 86 will hereinafter be referred to as a firstcommunication part 86, the memory 75 will hereinafter be referred to asa first memory 75, and the controller 90 will hereinafter be referred toas a first controller 90. In addition, the control panel 610 of thedisplay device 600 will hereinafter be referred to as a second controlpanel 610, the communication part 620 will hereinafter be referred to asa second communication part 620, the memory 630 will hereinafter bereferred to as a second memory 630, and the controller 640 willhereinafter be referred to as a second controller 640.

The control panel 100 may interface with the user. Therefore, thecontrol panel 100 may be arranged at the front surfaces of therefrigerating chamber doors (21, 22) so that the user can easilyapproach and manipulate the control panel 100.

The control panel 100 may include the input part 110 for usermanipulation, and a display 140 for providing the user with necessaryinformation.

The input part 110 may receive a control command for the refrigerator 1c from the user, and the display 140 may display the mirroring screenimage for the display device 600 thereon.

The input part 110 and the display 140 may be incorporated with eachother. More particularly, the display 140 may include the display 190mounted to the front surfaces of the refrigerating chamber doors (21,22), and the input part 110 may include a touchscreen panel 112incorporated with the display 190 provided to the front surfaces of therefrigerating chamber doors (21, 22). In order to discriminate among thefirst to fourth displays (150, 160, 170, 180), the display 180 willhereinafter be referred to as the fifth display 190 according to theembodiment.

The first communication part 86 may control the refrigerator 1 c tocommunicate with the external device under the control of the firstcontroller 90. The first communication part 86 may receive a remotecontrol signal from the second communication part 620 of the displaydevice 600 under the control of the first controller 90, such that thefirst communication part 86 can connect the refrigerator 1 c to themobile terminal 600.

The first communication part 86 may include a wireless Ethernet, awireless LAN (WLAN), and a local area network (LAN) unit according toperformance and structure of the refrigerator 1 c.

The LAN unit may be wirelessly connected to an Access Point (AP) at anAP installed position according to a control signal of the controller90. The LAN unit may support IEEE1002.11x. Short-range communication ofthe LAN unit may include Bluetooth, Bluetooth low energy, infrared dataassociation (IrDA), Wireless Fidelity (Wi-Fi), Ultra Wideband (UWB),Near Field Communication (NFC), etc.

The first memory 75 may store various data, control programs orapplications to drive and control the refrigerator 1 c. Moreparticularly, the first memory 75 may store programs or applications(e.g., Kitchen TV application) to provide the fifth display 190 of therefrigerator 1 c with the mirroring screen image of the display device600. This application may be stored in the initial stage, or may bedownloaded from the user during operation of the refrigerator 1 c.

The first memory 75 may store a control program to provide the fifthdisplay 190 of the refrigerator 1 c with the mirroring screen image ofthe display device 600, User Interface (UI) related to the application,an object (e.g., image, text, icon, button, etc.) to provide the UI,user information, documents, DBs, or associated data.

The first controller 90 may control signal flow of the internalconstituent elements of the refrigerator 1 c and overall operation ofthe refrigerator 1 c, and may perform data processing.

If the user inputs a control command or a predetermined condition issatisfied, the first controller 90 may perform the control program orapplication stored in the memory 75.

The controller 90 may include a Read Only Memory (ROM) 94 for storingthe control program or application needed to control the processor 92and the refrigerator 1 c; and a Random Access Memory (RAM) 96 forstoring the external input signal or data of the refrigerator 1 c orused as a storage region corresponding to various tasks executed in therefrigerator 1 c. ROM 94 and RAM 96 of the controller 90 mayconceptually include ROM and RAM of the memory 75.

The embodiment of the present disclosure will hereinafter be describedwith reference to the attached drawings.

The refrigerator 1 c according to the embodiment may provide the userwith the mirroring screen image of the display device 600 through theapplication supporting the mirroring function. The user may receive themirroring screen image of the display screen image through the fifthdisplay 190.

FIGS. 46 to 48 are exemplary views illustrating the principal screenimages of the application supporting the mirroring function.

FIG. 46 is a view illustrating the device search screen image S19, FIG.47 is a view illustrating the channel search screen image of the displaydevice 600, and FIG. 48 is a view illustrating the set-top box (STB)screen image S21 for a single channel

Referring to FIG. 46, the device search screen image S19 may include atitle region R19, a subtitle region R20, a description region R21, asearch region R22, and a re-search button region R23.

The title region R19 may be provided at the uppermost part of the devicesearch screen image S19, and may display the name (e.g., “Kitchen TV”)of a current execution application.

The subtitle region R20 may be provided at the lower end of the titleregion R19, and may display the name (e.g., “Searched TV list”) of acurrent execution application.

The description region R21 may be located at the lower end of thesubtitle region R20, and may provide text or images for describingdetailed content of the current execution application screen image.

The search region R22 may be provided at the center of the device searchscreen image S19, and the search icon (I21) for indicating that therefrigerator 1 c is searching for the devices located in the vicinity ofthe refrigerator 1 c may be displayed on the search region R22. Thesearch icon (I21) may be provided in the form of images or animationimages.

The re-search button region R23 may be located at the lowermost part ofthe connection screen image, and the re-search button B16 forre-searching for the device may be provided in the re-search buttonregion R23. While the refrigerator 1 c is searching for the devicelocated in the vicinity of the refrigerator 1 c, the re-search buttonB16 may be deactivated. If the searching of the peripheral device iscompleted, the re-search button B16 may be activated.

If the searching of the peripheral device of the refrigerator 1 c iscompleted, the device search screen image S19 of FIG. 46 may be switchedto the channel search screen image S20 of FIG. 47.

Referring to FIG. 47, the channel search screen image may include atitle region R19, a subtitle region R20, a description region R21, adevice list display region R23, and a re-search region R22. Inassociation with the title region R19, the subtitle region R20, thedescription region R21, and the re-search region R22, the samedescription as in FIG. 47 will herein be omitted for convenience ofdescription.

The images and titles of the searched devices may be displayed on thedevice list display region R23. If the plurality of devices is searched,all the corresponding devices may be displayed. In accordance with theembodiment, if the devices are not searched, the message indicating thatno device is searched may be displayed.

If the user touches at least one of the device lists of the channelsearch screen image S20, the device search screen image S19 of FIG. 47may be switched to the set-top box (STB) screen image S21 of FIG. 48.

Referring to FIG. 48, the STB screen image S21 may include a titleregion R18, a subtitle region R20, a screen display region R24, a deviceinformation provision region R25, and a control region R26. Inaccordance with the embodiment, the channel search screen image S20 mayalso display the return icon (I22) for returning to the channel searchscreen image S20. In association with the title region R19, the samedescription as in FIG. 47 will herein be omitted for convenience ofdescription.

The subtitle region R20 may be provided at the lower end of the titleregion R19, and the current provision channel information (e.g., achannel number, a channel title, etc.) may be displayed on the subtitleregion R20.

The screen display region R24 may be provided at the lower end of thesubtitle region R20, and may provide the mirroring screen image for thedisplay device 600. In other words, the same screen image as the currentdisplay device 600 may also be displayed on the screen display regionR24.

The device information provision region R25 may be provided at thecenter of the STB screen image S21, and may display informationregarding the display device 600 connected to the refrigerator 1 c. Forexample, although the image of the display device 600 and the name ofthe connected display device 600 may be displayed on the deviceinformation provision region R25, and the scope of exemplary displayableinformation is not limited thereto.

The control region R26 may be provided at the lower end of the STBscreen image S21, and the control tool for controlling the mirroringscreen image of the screen display region may be provided in the controlregion. The user may control the on or off operation of the mirroringscreen image using the control tool, or may control the channel orvolume. In the meantime, the scope of the control region R26 is notlimited to the above-mentioned example, and various control methods canbe made available according to the design of the control tool.

Although the embodiments of the present invention have been disclosedfor illustrative purposes, those skilled in the art will appreciate thatvarious modifications, additions and substitutions are possible, withoutdeparting from the scope and spirit of the invention as disclosed in theaccompanying claims.

What is claimed is:
 1. A refrigerator comprising: a proximity sensor; adisplay; a microphone; a main body including at least one or morechambers; at least one chamber door including the proximity sensor, thedisplay and the microphone, and coupled to the main body via a hinge;and at least one processor configured to: based on a first predeterminedcondition being detected by the proximity sensor while the refrigeratoris in a voice recognition deactivation state and the display is turnedoff, control the display to display a first screen of a voice controlscreen, the first screen indicating a voice recognition preparation stepof a voice recognition activation state, based on a first voice commandbeing received by the microphone while the refrigerator is in the voicerecognition activation state and the first screen being displayed,control the display to display a second screen of the voice controlscreen, the second screen indicating a voice processing step of thevoice recognition activation state and being different from the firstscreen, perform a function associated with the first voice command,after performing the function associated with the first voice command,switch from the voice recognition activation state to the voicerecognition deactivation state, based on a predetermined voice signalbeing received by the microphone while the refrigerator is in the voicerecognition deactivation state, control the display to display the firstscreen indicating the voice recognition preparation step of the voicerecognition activation state, based on a second voice command beingreceived by the microphone while the refrigerator is in the voicerecognition activation state, control the display to display the secondscreen indicating the voice processing step of the voice recognitionactivation state, and based on a second predetermined condition beingsatisfied, control the display to display temperature information of atleast one of the one or more chambers of the main body.
 2. Therefrigerator of claim 1, wherein the first predetermined conditioncomprises detecting an initiation of access to the refrigerator in asensing region of the proximity sensor.
 3. The refrigerator of claim 1,wherein the second screen includes images of text and an animationaccording to a progress of the voice processing step.
 4. Therefrigerator of claim 1, wherein the at least one processor is furtherconfigured to: after controlling the display to display the secondscreen indicating the voice processing step, control the display todisplay a third screen indicating a result providing step of the voicerecognition activation state.
 5. The refrigerator of claim 1, whereinthe second predetermined condition comprises detecting a change in astate of the at least one chamber door, and wherein the state of the atleast one chamber door including an open state or a closed state.
 6. Therefrigerator of claim 1, wherein the at least one processor is furtherconfigured to: based on a voice recognition deactivation command beingreceived while the refrigerator is in the voice recognition activationstate, switch from the voice recognition activation state to the voicerecognition deactivation state.
 7. The refrigerator of claim 6, whereinthe voice recognition deactivation command includes a state change ofthe at least one chamber door.
 8. The refrigerator of claim 1, furthercomprising: a control panel including at least one button at a frontsurface of the at least one chamber door, wherein the display isintegrated into the control panel and the at least one button isdisposed separately from the display.
 9. The refrigerator of claim 1,wherein the at least one processor is further configured to: based on aninput for changing a power consumption parameter being received, applyan energy saver function to change a power consumption pattern of therefrigerator.
 10. The refrigerator of claim 1, wherein the one or morechambers of the main body include an upper chamber and a lower chamber.11. The refrigerator of claim 10, wherein the at least one chamber doorincludes a first door and a second door associated with the upperchamber and a third door associated with the lower chamber.
 12. Therefrigerator of claim 11, wherein the proximity sensor, the display, andthe microphone are disposed in the second door associated with the upperchamber.
 13. The refrigerator of claim 12, wherein the first door isdisposed on a left side of the upper chamber and the second door isdisposed on a right side of the upper chamber.
 14. The refrigerator ofclaim 1, wherein the at least one processor is further configured to:control the display to display information associated with a temperatureof the at least one chamber.
 15. The refrigerator of claim 14, whereinthe display information associated with the temperature of the at leastone chamber includes information for setting a temperature of the atleast one chamber.
 16. The refrigerator of claim 1, wherein the at leastone processor is further configured to: in a case that a predeterminedtime has elapsed, control the display to turn off.
 17. The refrigeratorof claim 1, wherein the at least one processor is further configured to:in response to a third predetermined condition being detected, controlthe display to turn on.
 18. The refrigerator of claim 17, wherein thethird predetermined condition includes at least one of detecting achange of state of the at least one chamber door to an open position,detecting an input associated with the at least one chamber door, ordetecting an input using the proximity sensor.
 19. The refrigerator ofclaim 1, wherein the voice control screen includes a visual indicatorcomprising at least one of text or animation image indicating that therefrigerator is in the voice recognition preparation step.
 20. Therefrigerator of claim 19, wherein the second screen includes a visualindicator comprising at least one of text or animation indicating thatthe refrigerator is in the voice processing step.